Yin Sun

In modern computer systems, long-running jobs are divided into a large number of short tasks and executed in parallel. As the system size and complexity scales up, the task service times become highly random and the job service time is constrained by the slowest straggling tasks. An efficient solution is to replicate a task on multiple servers and wait for one task copy to finish early. However, delay-optimal scheduling of replications and more general coding techniques is notoriously difficult, with little understanding so far. In this research, low-complexity scheduling policies are proposed for queueing systems with replications, and are proven to be delay-optimal or near delay-optimal among all causal and non-preemptive policies. These results are established for general system settings and delay metrics which allow for arbitrary arrival processes, arbitrary job sizes, arbitrary soft deadlines, and heterogeneous servers with data locality constraints. In order to prove these results, novel sufficient conditions are developed for sample-path delay optimality and near delay optimality, which are general and can be applied to any queueing system. Applications of this study include cloud computing, cloud storage, machine learning, and many Web services.